Multi-frame Super Resolution Based on Block Motion Vector Processing and Kernel Constrained Convex Set Projection

Even though substantial progress has been made in super resolution research, many issues regarding robust sub-pixel estimation and fast implementation of feature preserving restoration still exist. To obtain more reliable sub-pixel information, we proposed to correct mis-aligned sub-pixels by motion vector (MV) processing based on hierarchical block partition and weighted vector median filtering (WVMF). Two indices – relative displaced frame difference and motion vector similarity degree – are computed and compared with trained thresholds to classify the motion blocks into reliable and unreliable groups. Then the unreliable blocks are divided into four sub-blocks with their motion vector processed by WVMF based on the reliability information of their neighborhood blocks. To preserve the local features such as edge direction, strength as well as its spread region, anisotropic kernels are learned from local gradient fields to represent edge information. Finally, a kernel constrained projection is established for restoring high resolution frames. The experimental results show that the proposed algorithm preserves important features in the images and outperforms the traditional POCS method.